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Societal  Problems 


Climate  change  and  the  environment 
Powering  our  civilization 
Disease,  epidemics,  and  health  care 
Livable  megacities 
Safety  and  security 
Transportation 


Society’s  Dependence  on  Software 
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Trend  Toward  Increasing  Scale-1 


Research  Areas 


Enormous  web  service  and 
computing  infrastructure 

Supply  chain  systems 

Software-based  engineering 
systems 
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Trend  Toward  Increasing  Scale-2 


Healthcare  Infrastructure 
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Trend  Toward  Increasing  Scale-3 


Homeland  Security 


Ultra-Large-Scale  Systems 

-r— Software  Engineering  Institute 

Carnegie  Mellon 

Linda  Northrop:  March  6,  2008 

6 

©2008  Carnegie  Mellon  University 

Trend  Toward  Increasing  Scale-4 


Networked  Automobiles 
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Trend  Toward  Increasing  Scale-5 


Saving  the  Environment 
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Increasing  Scale  In  Military  Systems 


Increasingly  Complex  Systems 

•  ultra-large,  network-centric,  real-time, 
cyber-physical-social  systems 

—  thousands  of  platforms,  sensors,  decision 
nodes,  weapons,  and  warfighters 

—  connected  through  heterogeneous  wired 
and  wireless  networks 


Goal:  Information  Superiority 

•  Transient  and  enduring  resource 
constraints  and  failures 

•  Continuous  adaptation 

—  changes  in  mission  requirements 

—  changes  in  operating  environments 

—  changes  in  force  structure 

—  perpetual  systems’  evolution 

—  addition  of  new  systems 

•  Sustainable  -  legally,  technically,  politically 
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The  Challenge 


The  Honorable 
Claude  M.  Bolton,  Jr. 


Software  Engineering  Institute 


“Our  soldiers  depend  on  software  and  will 
depend  more  on  software  in  the  future. 

The  Army’s  success  depends  on  software 
and  the  software  industry. 

We  need  better  tools  to  meet  future 
challenges,  and  neither  industry  nor 
government  is  working  on  how  to  do  things 
light-years  faster  and  cheaper. 

How  can  future  systems,  which  are  likely  to 
be  a  billion  lines  of  code,  be  built  reliably 
if  we  can’t  even  get  today’s  systems  right?” 

—  Asst  Sec  Army  Claude  Bolton 
August  16,  2005 


Carnegie  Mellon 
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Ultra-Large-Scale  (ULS)  Systems  Study 


Gather  leading  experts  to  study: 

•  characteristics  of  ULS  systems 

•  challenges  and  breakthroughs  required 

•  promising  research  and  approaches 

Intended  outcomes: 

•  ULS  System  Research  Agenda 

•  program  proposal 

•  collaborative  research  network 


About  the  Effort 

Funded  by  the  Army  (ASA  ALT) 

Staffing:  9  member  SEI  team 

13  member  expert  panel 

Duration:  one  year  (04/05  -  05/06) 
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SEI  Team 


Linda  Northrop  Rick  Kazman 

Software  Design  and  Architecture,  Adaptive  Architectures,  Design  Methods 

Software  Product  Lines 


Peter  Feiler  Mark  Klein 

Methodologies,  Real-time  Performance  Analysis, 

Configuration  Management  Software  Architecture  Design  and  Analysis 


John  Goodenough 

Software  Reliability, 
Safety  Assurance 


Mark  Pleszkoch 

Rigorous  Software  Engineering  Methods 


Rick  Linger  Kurt  Wallnau 

Rigorous  Software  and  Software  Components, 

Systems  Engineering  Program  Generation,  Language  Semantics 


Tom  Longstaff 

Security  and  Survivability  Engineering 
in  Complex  Systems 


Bill  Pollack 

Chief  Editor 


Daniel  Pipitone 

Chief  Graphical  Designer 
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SEI  Team  (and  Expert  Panel) 


Gregory  Abowd 

Georgia  Institute  of  Technology 


Peter  Neumann 

SRI  International  Computer  Science  Laboratory 


Carliss  Baldwin 

Harvard  Business  School 


Douglas  Schmidt 

Vanderbilt  University 

Mary  Shaw 

Carnegie  Mellon  University 


Bob  Balzer 

Teknowledge  Corporation 


Gregor  Kiczales 

University  of  British  Columbia 


Dan  Siewiorek 

Richard  P.  Gabriel  Carnegie  Mellon  University 

Sun  Microsystems 

Kevin  Sullivan 

University  of  Virginia 


AM  Mili 

New  Jersey  Institute  of  Technology 


John  Lehoczky  Jack  Whalen 

Carnegie  Mellon  University  PARC 
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Instead:  A  Different  Kind  of  Study 


It  presents  an  overall  research  agenda  --  not  just  for  new  tools  or  a 
new  software  method  or  modest  improvements  in  today’s  approaches. 

It  is  based  on  the  challenges  associated  with  ultra-large  scale. 

It  focuses  on  the  future. 

It  involves  an  interdisciplinary  base. 

It  takes  a  fresh  perspective  on  the  development,  deployment, 
operation,  and  evolution  of  software-intensive  systems. 


\ 

Germs  of  these  ideas  are  present  today  in  small  research  pockets; 
these  efforts  are  currently  too  small  to  have  much  impact  on  next- 
generation  ULS  systems. 
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ULS  Systems  Research  Agenda 


Describes 

•  the  characteristics  of  ULS 
systems 

•  the  associated  challenges 

•  promising  research  areas 
and  topics 

Is  based  on  a  new  perspective 
needed  to  address  the 
problems  associated  with 
ultra-large-scale  systems. 
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Working  Inside  and  Outside  the  Box 
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Working  Inside  and 
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The  Journey 
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What  Is  an  Ultra-Large-Scale  (ULS)  System? 


A  ULS  System  has  unprecedented  scale  in  some  of  these  dimensions: 

•  Lines  of  code 

•  Amount  of  data  stored,  accessed,  manipulated,  and  refined 

•  Number  of  connections  and  interdependencies 

•  Number  of  hardware  elements 

•  Number  of  computational  elements 

•  Number  of  system  purposes  and  user  perception  of  these  purposes 

•  Number  of  routine  processes,  interactions,  and  “emergent  behaviors” 

•  Number  of  (overlapping)  policy  domains  and  enforceable  mechanisms 

•  Number  of  people  involved  in  some  way 

r  ^ 

ULS  systems  will  be  interdependent  webs  of  software-intensive  systems, 
people,  policies,  cultures,  and  economics. 

ULS  systems  are  systems  of  systems  at  internet  scale. 

V _ I _ J 
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Scale  Changes  Everything 


'Characteristics 


Characteristics  of  ULS  systems  arise  because  of  their  scale. 

•  Decentralization 

•  Inherently  conflicting,  unknowable,  and  diverse  requirements 

•  Continuous  evolution  and  deployment 

•  Heterogeneous,  inconsistent,  and  changing  elements 

•  Erosion  of  the  people/system  boundary 

•  Normal  failures 

•  New  paradigms  for  acquisition  and  policy 


These  characteristics  may  appear  in  today’s  systems  and  systems  of  systems, 
but  in  ULS  systems  they  dominate. 

These  characteristics  undermine  the  assumptions  that  underlie  today’s 
v software  engineering  approaches. _ 
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Today’s  Approaches 


The  Engineering  Perspective  -  for  large  scale  software-intensive  systems 

•  largely  top-down  and  plan-driven 

•  requirements/design/build  cycle  with  standard  well-defined  processes 

•  centrally  controlled  implementation  and  deployment 

•  inherent  validation  and  verification 

The  Agile  Perspective  -  proven  for  smaller  software  projects 

•  fast  cycle/frequent  delivery/test  driven 

•  simple  designs  embracing  future  change  and  refactoring 

•  small  teams  and  retrospective  to  enable  team  learning 

•  tacit  knowledge 


Today’s  approaches  are  based  on  perspectives  that  fundamentally  do  not  cope 
with  the  new  characteristics  arising  from  ultra-large  scale. 

The  mentality  of  looking  backward  doesn’t  scale. 
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A  New  Perspective  is  Required 


“The  older  is  not  always  a  reliable  model  for  the  newer,  the 
smaller  for  the  larger,  or  the  simpler  for  the  more 
complex... Making  something  greater  than  any  existing  thing 

necessarily  involves  going  beyond  experience.” 

Henry  Petroski 

Pushing  the  Limits:  New  Adventures  in  Engineering 
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Today  We  Build  “Buildings” 


Characteristics 
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We  Need  To  Think 


Characteristics 


“Cities  are  places  of  massive 
information  flows,  networks,  and 
conduits,  and  myriad  transitory 
information  exchanges.” 

Howard  Rheinegold:  Smart  Mobs 
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We  Need  to  Think  Ecosystem 


Challenges 


ivcxmi  \xni‘ cosymtm 


Diverse  users  with  complex  networked 
dependencies  and  intrinsic  adaptive 
behavior 

Has: 

•  Robustness  mechanisms:  achieving 
stability  in  the  presence  of  disruption 

•  Measures  of  health:  diversity, 
population  trends,  other  key  indicators 
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We  Need  to  Think  Socio-Technical 
Ecosystems 


Characteristics 


Socio-technical  ecosystems  include 
people,  organizations,  and  technologies 
at  all  levels  with  significant  and  often 
competing  interdependencies. 

•  There  will  be  competition  for 
resources. 

•  There  will  be  organizations  and 
participants  responsible  for  setting 
policies. 

•  There  will  be  organizations  and 
participants  responsible  for  producing 
ULS  systems. 


•  There  will  need  to  be  local  and  global 
indicators  of  health  that  will  trigger 
necessary  changes  in  policies  and  in 
element  and  system  behavior. 
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Why  a  New  Perspective? 


Characteristics 


There  are  fundamental  assumptions  that  underlie  today’s  software 
engineering  and  software  development  approaches  that  are 
undermined  by  the  characteristics  of  ULS  systems. 

There  are  challenges  associated  with  ULS  systems  that  today’s 
perspectives  are  very  unlikely  to  be  able  to  address. 


s  ' 

For  the  last  forty  years,  engineering  has  been  the  dominant  metaphor  for 
software  systems  creation. 

In  ULS  systems,  we  now  are  dealing  with  not  just  software  but  an  ecosystem  of 
people,  organizations,  governance,  social  interaction,  hardware,  and  software. 


Engineering  is  no  longer  the  dominant  metaphor. 
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ULS  Systems  vs  Today’s  Approaches  - 1 


Characteristics 

Today’s  assumptions  undermined 

Decentralized  control 

All  conflicts  must  be  resolved  and 
resolved  centrally  and  uniformly. 

Inherently  conflicting,  unknowable,  and  diverse 
requirements 

Requirements  can  be  known  in 
advance  and  change  slowly. 

Tradeoff  decisions  will  be  stable. 

Continuous  evolution  and  deployment 

System  improvements  are  introduced 
at  discrete  intervals. 

Heterogeneous,  inconsistent,  and 
changing  elements 

Effect  of  a  change  can  be  predicted 
sufficiently  well. 

Configuration  information  is  accurate  and 
can  be  tightly  controlled. 

Components  and  users  are  fairly 
homogeneous. 

Ultra-Large-Scale  Systems 

-r— Software  Engineering  Institute 

Carnegie  Mellon 

Linda  Northrop:  March  6,  2008 

28 

©2008  Carnegie  Mellon  University 

ULS  Systems  vs  Today’s  Approaches  -  2 


Characteristics 


Characteristics 

Today’s  assumptions  undermined 

Erosion  of  the  people/system  boundary 

People  are  just  users  of  the  system. 

Collective  behavior  of  people  is  not  of  interest. 

Social  interactions  are  not  relevant. 

Normal  failures 

Failures  will  occur  infrequently. 

Defects  can  be  removed. 

New  paradigms  for  acquisition  and  policy 

A  prime  contractor  is  responsible  for  system 
development,  operation,  and  evolution. 

Ultra-Large-Scale  Systems 
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Challenges 


ULS  systems  will  present  challenges 
in  three  broad  areas: 

•  Design  and  evolution 

•  Orchestration  and  control 

•  Monitoring  and  assessment 
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Design  and  Evolution 


Challenges 


Specific  challenges  in  ULS  system  design  and  evolution  stemming  directly 
from  the  characteristics  of  ULS  systems: 

•  Economics  and  industry  structure 

•  Social  activity  for  constructing  computational  environments 

•  Legal  issues 

•  Enforcement  mechanisms  and  processes 

•  Definition  of  common  services  supporting  the  ULS  system 

•  Rules  and  regulations 

•  Agility 

•  Handling  of  change 

•  Integration 

•  User-controlled  evolution 

•  Computer-supported  evolution 

•  Adaptable  structure 

•  Emergent  quality 


— Ultra-Large-Scale  Systems 

Software  Engineering  Institute  CamegieMellon  Linda  Northrop:  March  6,  2008 

©  2008  Carnegie  Mellon  University 


Orchestration  and  Control 


Characteristics 


Orchestration  and  control  refers  to  the  set  of  activities  needed  to  make 
the  elements  of  a  ULS  system  work  together  in  reasonable  harmony  to 
ensure  continuous  satisfaction  of  mission  objectives. 

Orchestration  is  needed  at  all  levels  of  ULS  systems  and  challenges  us 
to  create  new  ways  for 

•  Online  modification 

•  Maintenance  of  quality  of  service  while  providing  necessary  flexibility 

•  Creation  and  execution  of  policies  and  rules 

•  Adaptation  to  users  and  contexts 

•  Enabling  of  user-controlled  orchestration 
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Monitoring  and  Assessment 


Characteristics 


The  effectiveness  of  ULS  system  design,  operation,  evolution,  orchestration, 
and  control  has  to  be  evaluated. 

There  must  be  an  ability  to  monitor  and  assess  ULS  system  state,  behavior, 
and  overall  health  and  well  being. 

Challenges  include 

•  Defining  indicators 

•  Understanding  why  indicators  change 

•  Prioritizing  the  indicators 

•  Handling  change  and  imperfect  information 

•  Gauging  the  human  elements 
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Where  Do  We  Focus  Our  Research 


•  Address  the  predominant 
characteristics  of  ULS  systems  and 
the  three  challenge  categories. 

•  Look  for  breakthroughs  not 
incremental  improvement  in  current 
approaches. 

•  Take  a  more  expansive  view  of 
software  research  and  include  its 
interactions  with  associated 
research  in  the  physical  and  social 
sciences. 
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Inspiration:  Open  Source  and  Cooperative 
Communities 


WikirhniA 
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Inspiration:  Game  Theory 


Research  Areas 


Oskar  Morgenstern 

1902-  1977 


Monograph  Efficiency 

9.3 

Marginal  Rale  of  Reference 

0.0 

Public  Pernepfirm  Indicator 

1.8 

Productivity  Potential  Index 

>24.0 

Expected  Utility 

1.5 

John  von  Neumann 


1903-  1957 


Monograph  Efficiency 

3  J 

Marginal  Rate  of  Reference 

12.0 

Public  Perception  Indicator 

22.3 

Productivity  Potential  Index 

-44,0 

Expected  Utility 

1.0 

John  F.  Nash  Jr. 


M928 

Monograph  Efficiency 

0.0 

Marginal  Rate  of  Reference 

0.0 

Public  Perception  Indicator 

11.1 

Productivity  Potential  Index 

+10.2 

Expected  Utility 

5.0 

Reinhard  Selten 


*1930 

Monograph  Efficiency 

6.3 

Marginal  Rate  of  Reference 

14.0 

Public  Perception  Indicator 

2.2 

Productivity  Potential  Index 

+  11.4 

Expected  Utility 

2-C 

Algorithmic  Mechanism  Design 

•  games  +  microeconomics  +  computation 

•  computational  markets  for  any  scarce 
ULS  resource? 


Institution  Design 

•  learning  games  +  self-reinforced 
expectations  +  cultural  norms 

•  better  formal  models  of  acquisition  in 
non-prime-dominated  landscape? 
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Inspiration:  Networks, 

Statistical  Mechanics,  Complexity 


\ 


Networks  Are  Everywhere 

Recurring  “scale  free”  structure 
•  internet  &  yeast  protein  structures 


Unstable  Equilibrium 

How  many  changes  before  a 
system  becomes  unstable? 


Analogous  dynamics?  What  scale  and  frequency  of 

•  epidemiology,  robustness  and  disruptions  can  be  expected? 

vulnerability 
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Economics  (Finance)  As  Design  Criteria 
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value 
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to  module  footprint 

Each  module  a  potential 
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Research  Portfolio 


We  recommend  an  interdisciplinary 
portfolio  of  seven  research  areas 
and  suggested  topics  for 
breakthrough  research  needed  to 
meet  the  challenges  associated  with 
ULS  systems. 

•  Is  not  expressed  in  terms  of 
today’s  “hot”  technologies. 

•  Does  not  supplant  current 
software  research. 

•  Expands  today’s  horizons. 
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Research  Areas  - 1 


Research  Areas 


6.1  Human  Interaction:  Involves 

anthropologists,  sociologists,  and  social 
scientists  conducting  detailed  socio- 
technical  analyses  of  user  interactions  in 
the  field,  with  the  goal  of  understanding 
how  to  construct  and  evolve  such  socio- 
technical  systems  effectively. 

•  Context-Aware  Assistive  Computing 

•  Understanding  Users  and  Their  Contexts 

•  Modeling  Users  and  User  Communities 

•  Fostering  Non-Competitive  Social 
Collaboration 

•  Longevity 
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Research  Areas  -  2 


6.2  Computational  Emergence: 

Explores  the  use  of  methods  and 
tools  based  on  economics  and  game 
theory  (e. g.,  mechanism  design)  to 
ensure  globally  optimal  ULS  system 
behavior  by  exploiting  the  strategic 
self  interests  of  the  system’s 
constituencies;  explores 
metaheuristics  and  digital 
evolution  to  augment  the  cognitive 
limits  of  human  designers. 

•  Algorithmic  Mechanism  Design 

•  Metaheuristics  in  Software 
Engineering 

•  Digital  Evolution 
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Research  Areas  -  3 


Research  Areas 


6.3  Design:  Broadens  the  traditional 
technology-centric  definition  of  design 
to  include  people  and  organizations; 
social,  cognitive,  and  economic 
considerations;  and  design  structures 
such  as  design  rules  and  government 
policies. 

•  Design  of  All  Levels 

•  Design  Spaces  and  Design  Rules 

•  Harnessing  Economics  to  Promote 
Good  Design 


•  Design  Representation  and  Analysis 


•  Assimilation 


•  Determining  and  Managing 
Requirements 
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Research  Areas  -  4 


6.4  Computational  Engineering:  Focuses  on 
evolving  the  expressiveness  of 
representations  to  accommodate  the 
semantic  diversity  of  many  languages  and 
focuses  on  providing  automated  support  for 
computing  the  evolving  behavior  of 
components  and  their  compositions. 

•  Expressive  Representation  Languages 

•  Scaled-Up  Specification,  Verification,  and 
Certification 

•  Computational  Engineering  for  Analysis  and 
Design 
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Research  Areas  -  5 


Research  Areas 


6.5  Adaptive  System  Infrastructure:  Investigates  integrated 

development  environments  and  runtime  platforms  that  will  support 
the  decentralized,  “always-on,”  nature  of  ULS  systems  as  well  as 
technologies,  methods,  and  theories  that  will  enable  ULS  systems 
to  be  developed  in  their  deployment  environments. 

•  Decentralized  Production  Management 

•  View-Based  Evolution 

•  Evolutionary  Configuration  and  Deployment 

•  In  Situ  Control  and  Adaptation 
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Research  Areas  -  6 


Research  Areas 


6.6  Adaptable  and  Predictable  System  Quality:  Focuses  on  how  to 
maintain  quality  in  a  ULS  system  in  the  face  of  continuous  change, 
ongoing  failures,  and  attacks  and  how  to  identify,  predict,  and 
control  new  indicators  of  system  health  (akin  to  the  U.S.  gross 
domestic  product)  that  are  needed  because  of  the  scale  of  ULS 
systems. 

•  Robustness,  Adaptation,  and  Quality  Attributes 

•  Scale  and  Composition  of  Quality  Attributes 

•  Understanding  People-Centric  Quality  Attributes 

•  Enforcing  Quality  Requirements 

•  Security,  Trust,  and  Resiliency 

•  Engineering  Management  at  Ultra-Large  Scales 
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Research  Areas  -  7 


6.7  Policy,  Acquisition,  and 
Management:  Focuses  on 
transforming  acquisition  policies 
and  processes  to  accommodate 
the  rapid  and  continuous  evolution 
of  ULS  systems  by  treating 
suppliers  and  supply  chains  as 
intrinsic  and  essential  components 
of  a  ULS  system. 

•  Policy  Definition  for  ULS  Systems 

•  Fast  Acquisition  for  ULS  Systems 

•  Management  of  ULS  Systems 
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Research  Areas 
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Research  Areas  and  Challenges 


Relationship  Between  Research  Areas  and  Challenges 


Research  Areas 

Design  and 
Evolution 

Orchestration 
and  Control 

Monitoring  and 
Assessment 

Human  Interaction 

• 

• 

Computational  Emergence 

• 

• 

Design 

• 

Computational  Engineering 

• 

Adaptive  System  Infrastructure 

• 

• 

Adaptable  and  Predictable  System  Quality 

• 

• 

• 

Policy,  Acquisition,  and  Management 

• 

• 
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Toward  a  Roadmap  for  a  ULS  Systems 
Research  Program 


There  are  many  possible  approaches  to 
structuring  a  research  program  from  the 
ULS  Systems  Research  Agenda 

We  provide  three  possible  support 
structures  based  on 

1.  Specific  DoD  missions  and  capabilities 

2.  DoD  research  funding  types  required 

3.  Estimates  of  the  relative  starting  points  o 
the  research 

Sponsors  with  different  needs  can 
choose  to  support  different  combinations 
of  research. 


Table  4:  Research  Areas  and  Range  of  Risk/Reward 


Research  Areas  and  Topics 

Existing 

Groundwork 

Breaking 

Ground 

New 

Direction 

Human  Interaction 

6. 1 ,  'i  Context-Aware  Assistive  Computing 

• 

6.1.2  Understanding  Users  and  Their  Contexts 

• 

• 

6-1-3  Modeling  Users  end  User  Communities 

• 

6.1  A  Fostering  Noncompetitive  Social  Collaboration 

• 

Q 

6-1.5  Longevity 

• 

Q 

Q 

Computational  Emergence 

6.2,1  Algorithmic  Mechanism  Design 

• 

Q 

• 

6.2.2  Meta  heuristics  in  Software  Engineering 

• 

Q 

6.2.3  Digital  Evolution 

• 

9 

Design 

6.3.1  Design  of  All  Levels 

• 

• 

Q 

6.3,2  Design  Spaces  and  Design  Rules 

• 

• 

6.3.3  Harnessing  Economics  to  Promote  Good  Design 

• 

• 

m 

6.3.4  Design  Representation  and  Analysis 

• 

Q 

6-3-5  Assimilation 

• 

• 

m 

6.3.6  Determ  i  ning  and  Ma n aging  Req  ui  rem  ents 

Computational  Engineering 

• 

• 

Q 

6.4.1  Expressive  Representation  Languages 

Q 

• 

• 

(  >v 

The  envisioned  outcome  of  the  proposed  research  is  a  spectrum  of  technologies  and 
methods  for  developing  ULS  systems,  with  national-security,  economic,  and  societal 

benefits  that  far  extend  beyond  ULS  systems  themselves. 

\ _ _ _ / 
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Study  Conclusions 


There  are  fundamental  gaps  in  our  current  understanding  of  software 
development  at  the  scale  of  ULS  systems. 

These  gaps 

•  present  profound  impediments  to  the  technically  and  economically 
effective  achievement  of  the  DoD  goals*  based  on  information 
superiority  and  to  effective  solutions  for  many  of  society’s  vexing 
problems 

•  require  a  broad,  fresh  perspective  and  interdisciplinary,  breakthrough 
research 

We  recommend 

•  a  ULS  Systems  Research  Agenda  that  includes  research  areas  based 
on  a  fresh  perspective  aimed  at  challenges  arising  from  increasing  scale 


As  stated  in  the  Quadrennial  Defense  Review  (QDR)  Report,  Feb  2006 
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ULS  Systems  Research  Study  Report 
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The  Start  of  a  Collaborative  Research  Network 


Workshops 


Keynotes 


Emerging 

Pane*s  Research 

Efforts 
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Activity  Since  Report  Published  in  May  06 


There  is  growing  community  interest  and  research  starts. 

Since  July  06 

•  more  than  95,000  downloads  of  the  report 

•  more  than  3,000  copies  of  the  report  distributed 

•  more  than  1 4  keynotes  and  more  than  25  presentations  by 
author  team 

•  three  press  and  one  industry  analyst  interviews 

•  research  workshops  at  OOPSLA  2006  and  ICSE  2007 

•  NSF  center  established 
New  SEI  Research  Activities 

•  Mechanism  design 

•  Implications  of  ULS  on  software  architecture 

Roadmap  Exercise  funded  by  Army  organization  (CERDEC) 

New  book  from  a  non-military  perspective  is  underway. 

SMART  Event 

Upcoming  event:  http://www.sei.cmu.edu/uls 

•  ICSE  2008  Workshop 
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Redesign  of  SEI  ULS  Systems  Website 


New  site  features  include 

•  Podcasts  and  video 
presentations 

•  ULS  Systems  news  & 
events 

—  RSS  feed 

•  ULS  Systems  library 

•  Online  Glossary 


http://www-sei.cmu.edu/uls/ 
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Work  with  Us 


ULS  Systems  Home 

Executive  Summary 

ULS  Systems  News*  RSS 

Focused  Research 

Human  Interaction 

Computational  Emergence 

Design 

Computational  Engineering 

ULS  systems  will  be  designed  beyond  human 

Adaptive  System  Infrastructure 

comprehension  by  design  methods  that 

Adaptable  and  Predictable 
System  Quality 

we  don't  completely  understand. 

Policy.  Acquisition,  and 

M  —  Richard  P.  Gabriel 

Management 

ULS  Systems  Resources 

ULS  Systems  Library 

Glossary 

Get  the  Report 

Contact 


Explore  the  Report 


Ultra-Large-Scale  Systems: 
The  Software  Challenge  of 
the  Future  Is  the  product  of  a 
12-month  study  of  ultra-targe- 
scale  (ULS)  systems 
software.  The  study  brought 
together  experts  In  software 
and  other  fields  to  answer  a 
question:  "Given  the  issues 
with  today's  software 
engmeenng.  how  can  we  build  the  systems  of  the 
future  that  are  likely  to  have  billions  of  lines  of 
code?"  The  report  details  a  broad,  multi¬ 
disciplinary  research  agenda  for  developing  the 
ultra-large-scale  systems  of  the  future. 


ULS  Systems  News 


Stay  informed  about  important 
discoveries  in  the  field,  new 
research  opportunities,  and  upcoming  ULS 
systems  events. 


What's  New 

Forum:  SMART  uitra-Laroe-Scaie  Systems 
March  6. 2008 


Workshop:  Second  International  Workshop  on 

yiga-large-Ssale  SgfSyyare-Ln.ten.siye  Systems 

(ULSSIS  2008):  May  10-11, 2008 


Presentation:  1 

Distributed  Real-Time  * 
lystems  with  QoS-Enabled 
Middleware  &  Model-Driven  Engineering 


nuo  onn 
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What  We  Learned  That  We  Want  to  Share 


>  There  is  an  unstoppable  trend  toward  increasing  scale  in  many 
systems  important  to  our  society. 

>  Scale  changes  everything. 

>  Manifestations  of  scale  and  its  attendant  complexity  arise  in  many 
disciplines,  and  can  be  understood  as  a  phenomenon  in  its  own 
right. 

>  New,  interdisciplinary  perspective  and  new  research  in  building 
ultra-large-scale  systems  is  long  overdue. 
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Contact  Information 


Linda  Northrop 

Director 

Product  Line  Systems  Program 
Telephone:  412-268-7638 
Email:  lmn@sei.cmu.edu 


U.S.  Mail: 

Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213-3890 

SEI  Fax:  412-268-5758 
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